Method of preprocessing image including biological information

ABSTRACT

A method of preprocessing an image including biological information is disclosed, in which an image preprocessor may set an edge line in an input image including biological information, calculate an energy value corresponding to the edge line, and adaptively crop the input image based on the energy value.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a divisional of and claims priority under 35 U.S.C.§§ 120/121 to U.S. patent application Ser. No. 15/049,529, filed on Feb.22, 2016, which claims priority under 35 U.S.C. § 119 to Korean PatentApplication No. 10-2015-0082693, filed on Jun. 11, 2015, in the KoreanIntellectual Property Office, the entire contents of each of which areincorporated herein by reference in their entirety.

BACKGROUND 1. Field

At least one example embodiment relates to a method of preprocessing animage including biological information.

2. Description of the Related Art

Recently, an importance of secure authentication is increasing due to adevelopment of various mobile devices such as smartphones and wearabledevices. Biometric based authentication technology may perform userauthentication using, for example, a fingerprint, an iris, a voice, aface, and blood vessels. Such biological characteristics used for theauthentication differ from individual to individual, rarely changeduring a lifetime, and have a low risk of being stolen or copied. Inaddition, individuals do not need to intentionally carry suchcharacteristics at all times and thus, may not suffer an inconvenienceusing the biological characteristics.

Currently, a fingerprint recognition method is most commonly used due toa high level of convenience, security, and economic efficiency. Thefingerprint recognition method may reinforce security of a user deviceand provide various application services, for example, mobile payment.

Sensors sensing biological information may be produced to have variousspecifications and thus, images to be output from individual sensors mayalso have various specifications. Thus, there is a desire for technologyfor preprocessing images of various specifications to register andrecognize a fingerprint.

SUMMARY

At least some example embodiments relate to an image preprocessingmethod.

According to at least some example embodiments an image preprocessingmethod includes obtaining an input image including biologicalinformation; setting at least one edge line in the input image;calculating at least one energy value corresponding to the at least oneedge line; and cropping the input image based on the at least one energyvalue.

The at least one energy value may include a variance of pixel valuesincluded in the at least one edge line.

The at least one edge line may include lines located on at least twodifferent edges in the input image.

The cropping may include performing a comparison operation based on theat least one energy value; detecting an edge line having a minimumenergy value based on a result of the comparison operation; and removingthe detected edge line. The cropping may include comparing the at leastone energy value to a threshold energy value; detecting an edge linehaving an energy value less than the threshold energy value based on aresult of the comparing; and removing the detected edge line.

The method may further include determining whether a size of the croppedinput image satisfies a first size condition; and setting at least onesecond edge line in the cropped input image, calculating at least onesecond energy value corresponding to the at least one second edge line,and additionally cropping the cropped input image based on the at leastone second energy value, in response to a determination that the size ofthe cropped input does not satisfy the first size condition.

The first size condition may include at least one of a condition that asize of the cropped input image is a first size; a condition that alength of any one side of the cropped input image is a first length; ora condition that at least one side of the cropped input image includes anumber of pixels, the number of the pixels being a power of 2.

The obtaining of the input image may include detecting a biologicalinput as a form of an analog signal; and obtaining the input image byconverting the analog signal to a digital image.

The method may further include enhancing the cropped input image using afrequency transform operation.

The biological information may include at least one of fingerprintinformation, blood vessel information, or iris information.

According to at least some example embodiments, an image preprocessingmethod may include obtaining an input image including biologicalinformation; setting a reference line in the input image; calculating areference energy value corresponding to the set reference line; settinga first line in the input image; calculating a first energy valuecorresponding to the set first line; and determining an effective areain the input image based on the reference energy value and the firstenergy value.

The reference line may include at least one line passing through acenter of the input image.

The setting of the first line may include setting the first line in anorder starting from any one edge of the input image to an opposite edgedirection, progressing iteratively to determine the effective area.

The reference energy value may include a variance of pixel valuesincludes in the reference line, and the first energy value includes avariance of pixel values includes in the first line.

The determining may include determining whether the first line isincluded in the effective area by performing a comparison operationbased on the energy value and the reference energy value. Thedetermining may include at least one of determining the first line to bea boundary of the effective area in response to the first energy valuebeing greater than the reference energy value; and setting a subsequentline in the input image, calculating a subsequent energy valuecorresponding to the subsequent line, and determining the effective areain the input image based on the reference energy value and thesubsequent energy value, in response to the first energy value beingless than the reference energy value.

The method may further include determining whether a first number ofboundaries is determined for the effective area; and setting a secondline for a boundary yet to be determined in the input image, calculatinga second energy value corresponding to the second line, and determiningthe effective area in the input image based on the reference energyvalue and the second energy value, in response to a determination thatthe number of boundaries is not determined for the effective area. Themethod may further include determining whether to receive a re-inputimage based on the effective area. The biological information mayinclude at least one of fingerprint information, blood vesselinformation, and iris information.

According to at least some example embodiments, a computer program isembodied on a non-transitory computer readable medium, the computerprogram being configured to control a processor to perform the method.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features and advantages of example embodiments ofthe inventive concepts will become more apparent by describing in detailexample embodiments of the inventive concepts with reference to theattached drawings. The accompanying drawings are intended to depictexample embodiments of the inventive concepts and should not beinterpreted to limit the intended scope of the claims. The accompanyingdrawings are not to be considered as drawn to scale unless explicitlynoted.

FIG. 1 illustrates an operation of obtaining an input image includingbiological information according to at least one example embodiment;

FIGS. 2 through 4B illustrate an operation of adaptively cropping aninput image according to at least one example embodiment;

FIG. 5 illustrates an example of a method of verifying biologicalinformation using an adaptive image cropping method according to atleast one example embodiment;

FIG. 6 illustrates an another example of a method of verifyingbiological information using an adaptive image cropping method accordingto at least one example embodiment;

FIGS. 7 and 8 illustrate an operation of determining an effective areain an input image according to at least one example embodiment;

FIG. 9 illustrates a method of registering and verifying biologicalinformation using an adaptive image cropping method and an effectivearea determining method according to at least one example embodiment;

FIGS. 10 and 11 are flowcharts illustrating an image preprocessingmethod according to at least one example embodiments; and

FIG. 12 illustrates an electronic system according to at least oneexample embodiment.

DETAILED DESCRIPTION

Detailed example embodiments of the inventive concepts are disclosedherein. However, specific structural and functional details disclosedherein are merely representative for purposes of describing exampleembodiments of the inventive concepts. Example embodiments of theinventive concepts may, however, be embodied in many alternate forms andshould not be construed as limited to only the embodiments set forthherein. Accordingly, while example embodiments of the inventive conceptsare capable of various modifications and alternative forms, embodimentsthereof are shown by way of example in the drawings and will herein bedescribed in detail. It should be understood, however, that there is nointent to limit example embodiments of the inventive concepts to theparticular forms disclosed, but to the contrary, example embodiments ofthe inventive concepts are to cover all modifications, equivalents, andalternatives falling within the scope of example embodiments of theinventive concepts. Like numbers refer to like elements throughout thedescription of the figures.

It will be understood that, although the terms first, second, etc. maybe used herein to describe various elements, these elements should notbe limited by these terms. These terms are only used to distinguish oneelement from another. For example, a first element could be termed asecond element, and, similarly, a second element could be termed a firstelement, without departing from the scope of example embodiments of theinventive concepts. As used herein, the term “and/or” includes any andall combinations of one or more of the associated listed items. It willbe understood that when an element is referred to as being “connected”or “coupled” to another element, it may be directly connected or coupledto the other element or intervening elements may be present. Incontrast, when an element is referred to as being “directly connected”or “directly coupled” to another element, there are no interveningelements present. Other words used to describe the relationship betweenelements should be interpreted in a like fashion (e.g., “between” versus“directly between”, “adjacent” versus “directly adjacent”, etc.).

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of exampleembodiments of the inventive concepts. As used herein, the singularforms “a”, “an” and “the” are intended to include the plural forms aswell, unless the context clearly indicates otherwise. It will be furtherunderstood that the terms “comprises”, “comprising,”, “includes” and/or“including”, when used herein, specify the presence of stated features,integers, steps, operations, elements, and/or components, but do notpreclude the presence or addition of one or more other features,integers, steps, operations, elements, components, and/or groupsthereof.

It should also be noted that in some alternative implementations, thefunctions/acts noted may occur out of the order noted in the figures.For example, two figures shown in succession may in fact be executedsubstantially concurrently or may sometimes be executed in the reverseorder, depending upon the functionality/acts involved.

Example embodiments of the inventive concepts are described herein withreference to schematic illustrations of idealized embodiments (andintermediate structures) of the inventive concepts. As such, variationsfrom the shapes of the illustrations as a result, for example, ofmanufacturing techniques and/or tolerances, are to be expected. Thus,example embodiments of the inventive concepts should not be construed aslimited to the particular shapes of regions illustrated herein but areto include deviations in shapes that result, for example, frommanufacturing.

Although corresponding plan views and/or perspective views of somecross-sectional view(s) may not be shown, the cross-sectional view(s) ofdevice structures illustrated herein provide support for a plurality ofdevice structures that extend along two different directions as would beillustrated in a plan view, and/or in three different directions aswould be illustrated in a perspective view. The two different directionsmay or may not be orthogonal to each other. The three differentdirections may include a third direction that may be orthogonal to thetwo different directions. The plurality of device structures may beintegrated in a same electronic device. For example, when a devicestructure (e.g., a memory cell structure or a transistor structure) isillustrated in a cross-sectional view, an electronic device may includea plurality of the device structures (e.g., memory cell structures ortransistor structures), as would be illustrated by a plan view of theelectronic device. The plurality of device structures may be arranged inan array and/or in a two-dimensional pattern.

at least some example embodiments Unless otherwise defined, all termsincluding technical and scientific terms used herein have the samemeaning as commonly understood by one of ordinary skill in the art towhich these example embodiments belong. It will be further understoodthat terms, such as those defined in commonly used dictionaries, shouldbe interpreted as having a meaning that is consistent with their meaningin the context of the relevant art and will not be interpreted in anidealized or overly formal sense unless expressly so defined herein.

Example embodiments to be described hereinafter may be used topreprocess an input image including biological information. For example,a preprocessing operation according to at least some example embodimentsmay be performed before a recognition operation is performed in afingerprint recognizer, a blood vessel recognizer, an iris recognizer,and the like. Hereinafter, an operation of recognizing biologicalinformation of a user may include an operation of authenticating oridentifying the user by recognizing the biological information of theuser.

At least some example embodiments may be provided in various forms ofproducts including, for example, a personal computer (PC), a laptopcomputer, a tablet PC, a smartphone, a television (TV), a smart homeappliance, an intelligent vehicle, a kiosk, and a wearable device. Forexample, at least some example embodiments may be applied to imagepreprocessing for user authentication in a smartphone, a mobile device,a smart home system, and the like. Similarly, at least some exampleembodiments may be applied to a payment service and an intelligentvehicle system through user authentication. At least some exampleembodiments may provide a result of user authentication robust against asituation in which a finger of a user is small or an input fingerprintis inaccurate. Hereinafter, at least some example embodiments will bedescribed in detail with reference to the accompanying drawings.

FIG. 1 illustrates an operation of obtaining an input image includingbiological information according to at least one example embodiment. Forease of description hereinafter, the biological information is assumedto be a fingerprint. However, various sets of biological informationrecognizable in a form of an image of blood vessels, an iris, and thelike may be used to provide the biological information.

Referring to FIG. 1, a sensor 110 senses a fingerprint input from auser. For example, the sensor 110 may include a plurality of sensingelements. The sensing elements may be arranged in a form of an array orin a matrix structure. The sensor 110 senses the input fingerprint,hereinafter also referred to as a fingerprint input, in a form of ananalog signal using the sensing elements. The sensor 110 converts thesensed analog signal to a digital image using an analog-to-digitalconverter (ADC). Hereinafter, the term “input image”, as used herein,will refer to the digital image obtained through the conversion.

Here, a size of the sensor 110 may differ from a size of a finger 120 ofthe user. A specification of the sensor 110 may be different frommanufacturers producing the sensor 110, and a size of a finger may bedifferent from users. For example, in a case of a user having a smallfinger, for example, a female user and a child, information irrelevantto a fingerprint may be sensed by the sensor 110. Also, noiseinformation may be sensed by the sensor 110 due to foreign substancessuch as perspiration. Also, fingerprint information may be inaccuratelyinput in various situations, for example, when a fingerprint is input onthe move. In such a case, a portion that does not correspond to thefingerprint, which is also referred to as a non-fingerprint area, may beincluded in an input image. Although described in detail hereinafter,example embodiments may provide technology for removing, from an inputimage, such a non-fingerprint area which does not correspond to afingerprint.

In addition, since a specification of the sensor 110 differs based onmanufacturers producing the sensor 110, the fingerprint may not bereadily recognized using a same recognizer in a fingerprint recognitionapplication. The specification may include a size of a sensor array, asensor resolution, and the like. The size of a sensor array refers to asize of a sensed area to be formed by the sensing elements. For example,the size of a sensor array may be 1 centimeter (cm) in height×3 cm inwidth. The sensor resolution indicates a number of the sensing elementsper unit length or unit area. When the sensor 110 has a higherresolution despite the same size of the sensor array, a greater numberof sensing elements may be included in the sensor 110. For example, thesensor resolution may be 56 pixels in height×144 pixels in width.

Although described in detail, example embodiments described herein mayprovide technology for recognizing a fingerprint without correcting arecognizer despite different sensor specifications by cropping an inputimage into an image of a preset or, alternatively, desired size. Thepreset or, alternatively, desired size may refer to a size used in arecognizer being used. At least some example embodiments may providetechnology for increasing a fingerprint recognition rate by adaptivelycropping a portion including less fingerprint information in an inputimage.

In addition, at least some example embodiments may provide technologyfor determining whether an input image includes information sufficientfor fingerprint recognition by calculating an effective areacorresponding to a fingerprint in the input image. When the input imageis not determined to include the information sufficient for fingerprintrecognition, a fingerprint recognition application may request a user tore-input a fingerprint, rather than recognizing or registering thefingerprint using the input image.

Further, at least some example embodiments may provide technology forimproving a fingerprint authentication performance by processing aninput image when performing the fingerprint authentication using atouch-type fingerprint sensor. At least some example embodiments mayprovide technology for determining whether to use an input image forfingerprint authentication by evaluating a quality of the input image.

At least some example embodiments may provide technology for increasinga fingerprint recognition rate by removing a non-fingerprint areaunnecessary for fingerprint recognition. The technology may remove thenon-fingerprint area in a line unit and thus, a removed boundary may beconsistently indicated in a frequency spectrum. Thus, performancedeterioration may be prevented by removing the non-fingerprint area. Inaddition, the technology may effectively remove the non-fingerprint areairrespective of a quality of an input image by comparing energy valuescalculated from edge lines in the input image and detecting thenon-fingerprint area, instead of detecting the non-fingerprint areausing a fixed threshold value. At least some example embodiments mayprovide technology for preventing a decrease in a fingerprintrecognition rate without incorporating a fingerprint image includingless fingerprint information into registration data.

At least some example embodiments may provide technology for increasinga fingerprint recognition speed. For example, the technology may performan operation necessary for fingerprint recognition only on a croppedportion, in lieu of performing such an operation on an entire inputimage. The technology may adjust a size of an input image to be a powerof 2. When the size of the input image increases twofold, high-speedprocessing may be enabled to find registration information in atransform domain, and global image enhancement may be enabled only witha one-time two-dimensional (2D) fast Fourier transform (FFT).

FIGS. 2 through 4B illustrate an operation of adaptively cropping aninput image according to at least one example embodiment. The operationof adaptively cropping an input image may be performed by an imagepreprocessor according to an embodiment. The image preprocessor may beprovided in a form of a software module and implemented by at least oneprocessor. The software module may be recorded in a form of a program ina memory connected to the processor. Alternatively, the imagepreprocessor may be provided in a form of a hardware module.Alternatively, the image preprocessor may be provided in a form of acombination of the software module and the hardware module. In such acase, a function implemented by the software module may be performed bythe processor, and a function implemented by the hardware module may beperformed by a corresponding hardware. The processor and the hardwaremay mutually exchange a signal through an input and output bus.

Referring to FIG. 2, an input image 210 includes fingerprintinformation. As described in the foregoing, since a sensor specificationdiffers from manufacturers, a fingerprint recognition application maynot use a same recognizer. The image preprocessor may crop the inputimage 210 into an image of a preset or, alternatively, desired size andthus, allow the fingerprint recognition application to use an inputimage sensed through a sensor of various specifications without changinga recognizer.

When cropping an input image 210, the image preprocessor may determine aportion to be adaptively removed. As illustrated in FIG. 2, an upperleft portion and a right portion in the input image 210 do not includethe fingerprint information. The image preprocessor generates a resultimage 220 by adaptively cropping the input image 210. The result image220 has the preset or, alternatively, desired size, and includes amaximum or, alternatively, increased amount of effective information inthe size that may be used for fingerprint recognition. Thus, afingerprint recognition rate may be improved.

The result image 220 is provided to a device 240 operating thefingerprint recognition application. The result image 220 may be usedfor fingerprint registration or fingerprint recognition. For ease ofdescription, the image preprocessor is illustrated to operate separatelyfrom the device 240. However, the image preprocessor may be included inthe device 240. For example, the image preprocessor may operate by atleast one processor or at least one other hardware-implemented deviceincluded in the device 240, or a combination thereof.

Hereinafter, the operation of adaptively cropping the input image 210will be described in detail with reference to FIG. 3. Referring to FIG.3, the image preprocessor sets a plurality of edge lines, for example, afirst edge line 310 and a second edge line 320, in the input image 210.An edge line refers to a line located at an edge in an input image. Forexample, when the input image 210 is provided in a quadrangular form, anedge line may be located at a left edge, a right edge, an upper edge, ora lower edge.

An input image may include a plurality of pixels. The pixels maycorrespond to sensing elements in a sensor. For example, a pixel valueof each pixel may indicate information sensed by a corresponding sensingelement (e.g., brightness).

An edge line may include a plurality of edge pixels located at an edgeof an input image. The image preprocessor may calculate an energy valuecorresponding to an edge line based on pixel values of the edge pixelsincluded in the edge line. For example, when an edge line includes agreater amount of fingerprint information, an energy value correspondingto the edge line may be calculated to be greater. Conversely, when anedge line includes a lower amount of fingerprint information, an energyvalue corresponding to the edge line may be calculated to be lower.

The image preprocessor may calculate such an energy value using a presetor, alternative, desired energy function. For example, the preset or,alternative, desired energy function may refer to a function used tocalculate a variance of the pixel values of the edge pixels included inthe edge line. According to an embodiment, the energy function may notbe limited to the function for calculating the variance. The energyfunction may be modified to be a function for outputting a valueproportional to an amount of fingerprint information included in theedge line. For example, the energy function may be a function forcalculating a mean of the pixel values of the edge pixels included inthe edge line.

In one example, the image preprocessor sets the first edge line 310corresponding to the left edge in the input image 210. The imagepreprocessor calculates a first energy value “v1” corresponding to thefirst edge line 310. The image preprocessor crops the input image 210based on the v1. For example, the image preprocessor may compare the v1to a threshold energy value, and remove the first edge line 310 inresponse to the v1 being less than the threshold energy value. Thethreshold energy value may be predetermined or, alternatively,determined in response to a standard for determining usability offingerprint information included in an edge line. The image preprocessordetermines whether an amount of fingerprint information included in thefirst edge line 310 set in the input image 210 satisfies the standard,and adaptively crops the input image 210.

In another example, the image preprocessor sets the first edge line 310corresponding to the left edge in the input image 210 and the secondedge line 320 corresponding to the right edge in the input image 210.The image preprocessor calculates the v1 corresponding to the first edgeline 310 and a second energy value “v2” corresponding to the second edgeline 320.

The image preprocessor crops the input image 210 based on the v1 and thev2. For example, the image preprocessor may compare the v1 to the v2,and detect an edge line including a lower amount of fingerprintinformation. Here, the first edge line 310 includes a greater number ofedge pixels including the fingerprint information than the second edgeline 320. In such a case, the v1 may be calculated to be greater thanthe v2. The image preprocessor may detect the second edge line 320 to bethe edge line including the lower amount of the fingerprint informationbecause the v2 is less than v1. The image preprocessor adaptively cropsthe input image 210 by removing the detected second edge line 320.

The image preprocessor may determine whether a size of the cropped inputimage satisfies a predetermined or, alternatively, desired sizecondition. The predetermined or, alternatively, desired size conditionmay be variously set to be a condition of which the size of the croppedinput image corresponds to a preset or, alternatively, desired size, acondition of which a length of any one side of the cropped input imagecorresponds to a preset or, alternatively, desired length, or acondition of which at least one side of the cropped input image includespixels. Here, a number of the pixels corresponds to a power of 2. A sizeof the input image 210 may be expressed as a size of a pixel array. Forexample, the size of the input image 210 may be 56 pixels in height×144pixels in width.

In example embodiments described with reference to FIGS. 2 through 4,adaptive cropping may be performed only on a length in a horizontaldirection. For example, the preset or, alternatively, desired size maybe 56 pixels in height×128 pixels in width. In such an example, theimage preprocessor may adaptively remove 16 lines in the horizontaldirection of the input image 210 to crop the input image 210 to be thepreset or, alternatively, desired size of 56 pixels in height×128 pixelsin width. The adaptive cropping may not be limited to the horizontaldirection and thus, such an operation may be applicable to a verticaldirection and a lateral and vertical direction.

When the size of the cropped input image is determined to beunsatisfactory for the predetermined or, alternatively, desired sizecondition, the image preprocessor may iterate the operations describedin the foregoing. For example, the image preprocessor may iterate theoperations 16 times to crop the input image 210 of the size of 56 pixelsin height×144 pixels in width to be the preset or, alternatively,desired size of 56 pixels in height×128 pixels in width. Thus, theresult image 220 may be generated as illustrated in FIG. 2.

Referring back to FIG. 3, when the first edge line 310 is removed at acurrent iteration, the first edge line 310 may be set to be a subsequentline in a right direction at a subsequent iteration. When the secondedge line 320 is removed at the current iteration, the second edge line320 may be set to be a subsequent line in a left direction at thesubsequent iteration.

Referring to FIGS. 4A and 4B, a graph 420 of FIG. 4B indicates adistribution of energy of lines distinguished along a horizontaldirection in an input image 410 of FIG. 4A. For example, the lines inthe input image 410 may have different horizontal direction components,for example, x-axis values. Referring to the graph 420, an area withoutfingerprint information has a relatively low energy value, and an areahaving fingerprint information has a relatively high energy value. Thus,the image preprocessor may adaptively crop the input image 410 byremoving lines having an energy value less than a predetermined or,alternatively, desired value.

An operation of the image preprocessor may be implemented as an adaptiveimage cropping (AIC) algorithm illustrated in Table 1. The AIC algorithmmay iteratively remove, from an input image in a line unit, a column ora row in an upper, a lower, a left, and a right edge portion.

TABLE 1 1. Initalize start and end point    n_start = 1, m_start=1   n_end=N, m_end = M 2. find minimum projected energy value at fouredges    min_value = min{f_(x)(n_start), f_(x)(n_end), f_(y)(m_start)f_(y)(m_end)} 3. update edge index that shows the minimum value.   Update n_start ++ or m_start++ or n_end − or m_end −− 4. comparemin_value with a threshold, then smaller than the threshold go to step2, or go to 5 5. crop the given image with the given four crop indexes

Referring to Table 1, in a first operation of the AIC algorithm, theimage preprocessor initializes indices of edge lines corresponding tofour edges in an input image. Here, “n_start” denotes an index of anedge line corresponding to a left edge, and “n_end” denotes an index ofan edge line corresponding to a right edge. “m_start” denotes an indexof an edge line corresponding to an upper edge, and “m_end” denotes anindex of an edge line corresponding to a lower edge. A size of the inputimage may be M pixels in height×N pixels in width.

In a second operation of the AIC algorithm, the image preprocessorcalculates energy values of all the edge lines corresponding to the fouredge lines, and extracts a minimum energy value (or, alternatively, anenergy value below a desired threshold) “min_value” from the calculatedenergy values. “fx( )” denotes a function for calculating an energyvalue of an edge line in a vertical direction, for example, a left edgeline and a right edge line, and “fy( )” denotes a function forcalculating an energy value of an edge line in a horizontal direction,for example, an upper edge line and a lower edge line.

In a third operation of the AIC algorithm, the image preprocessorupdates an index of the edge line having the minimum energy value (or,alternatively, an energy value below a desired threshold). For example,when the left edge line has the min_value, the image preprocessor mayincrease the n_start. When the right edge line has the min_value, theimage preprocessor may decrease the n_end. Similarly, when the upperedge line has the min_value, the image preprocessor may increase them_start. When the lower edge line has the min_value, the imagepreprocessor may decrease the m_end.

In a fourth operation of the AIC algorithm, the image preprocessorcompares the min_value to a threshold value. In response to themin_value being less than the threshold value as a result of thecomparing, the image preprocessor returns to the second operation of theAIC algorithm and iteratively performs the operation. In response to themin_value being greater than or equal to the threshold value, the imagepreprocessor performs a fifth operation of the AIC algorithm.

In the fifth operation of the AIC algorithm, the image preprocessorcrops the input image based on the four edge indices. The four edgeindices may be also referred to as crop indices. When a size of thecropped image obtained by cropping the input image is less than or equalto a preset or, alternatively, desired size, the image preprocessor maydetermine that the input image includes insufficient fingerprintinformation. In such a case, fingerprint authentication may not beperformed, and the image preprocessor may request a user to re-input afingerprint.

The AIC algorithm illustrated in Table 1 is provided as an example andthus, various modifications may be made to the AIC algorithm. In oneexample, as an alternative to comparing the min_value to the thresholdvalue in the fourth operation of the AIC algorithm, the imagepreprocessor may compare a difference between horizontal indices, forexample, a difference between the n_end and n_start, to a thresholdhorizontal length value, or compare a difference between verticalindices, for example, a difference between the m_end and m_start, to athreshold vertical length value. The threshold horizontal length valuemay correspond to a preset or, alternative, desired width at which theinput image is to be cropped, and the threshold vertical length valuemay correspond to a preset or, alternative, desired height at which theinput image is to be cropped.

When the difference between the horizontal indices reaches the thresholdhorizontal length value, only the vertical indices may be used in thesecond operation of the AIC algorithm, and only the vertical indices maybe updated in the third operation of the AIC algorithm. When thedifference between the vertical indices reaches the threshold verticallength value, only the horizontal indices may be used in the secondoperation of the AIC algorithm, and only the horizontal indices may beupdated in the third operation of the AIC algorithm.

FIG. 5 illustrates an example of a method of verifying biologicalinformation using an adaptive image cropping method according to atleast one example embodiment. Referring to FIG. 5, an image preprocessorperforms operations 520 and 530. In operation 520, the imagepreprocessor adaptively crops an input image. The descriptions providedwith reference to FIGS. 1 through 4B may be applied to the operation 520and thus, more detailed and repeated descriptions will be omitted here.

In operation 530, the image preprocessor enhances the cropped image. Forexample, the image preprocessor may enhance the cropped image using afrequency transform method. The image preprocessor may perform atwo-dimensional (2D) fast Fourier transform (FFT) on the cropped imageto obtain frequency domain information of the cropped image.

To improve a performance of the FFT, the image preprocessor may crop theinput image to have a length of a power of 2. The image preprocessor mayset, to correspond to a twofold increase, at least one of a width lengthand a height length of a preset or, alternatively, desired size at whichthe input image is cropped. For example, the image preprocessor may setthe size to be 56 pixels in height×128 pixels in width. In such anexample, the image preprocessor may form a size of the cropped image tobe a size of 128 pixels in height×128 pixels in width using a zeropadding method, and then perform the FFT.

The image preprocessor may enhance the image by applying variousfiltering methods to the frequency domain information. For example, theimage preprocessor may apply, to the frequency domain information, alow-pass filter or a band-pass filter. In operation 530, the imagepreprocessor may apply an image quality enhancement algorithm, otherthan an enhancement method based on the frequency transform method.

When the input image is an excessively high-resolution image, the imagepreprocessor performs image scaling in operation 510. The image scalingmay include image decimation. For example, the image preprocessor mayperform the image decimation by extracting a representative pixel valuefrom pixel values of neighboring 2×2 pixels. When an image resolution ishigh, an operation quantity may also increase. Thus, the imagepreprocessor may appropriately reduce the resolution of the input imagethrough the image decimation. The image preprocessor may perform theimage decimation by reducing a data quantity of an image sample tomaintain a quality greater than or equal to a preset or, alternatively,desired resolution.

The image decimation may not be limited to the method of extracting arepresentative pixel value, and various modifications may be made to theimage decimation, for example, performing low-pass filtering on pixelvalues of neighboring pixels.

A result image obtained through the operations 510 through 530 may betransferred to a biological information verifying application. Inoperation 540, the biological information verifying application verifiesbiological information by comparing the result image to registeredbiological information 550. The biological information verifyingapplication may use an image pattern matching-based fingerprintrecognizer. The image pattern matching-based fingerprint recognizer maymatch two images, for example, a result image and a registered image,using a frequency-based matching method. Although, for ease ofdescription, the image preprocessor and the biological informationverifying application are described to be separate, the imagepreprocessor may be included as an input end of the biologicalinformation verifying application.

FIG. 6 illustrates another example of a method of verifying biologicalinformation using an adaptive image cropping method according to atleast one example embodiment. Referring to FIG. 6, in operation 610, animage preprocessor adaptively crops an input image. In operation 620,the image preprocessor performs an FFT. In operation 630, the imagepreprocessor applies band-pass filtering to frequency domaininformation. As a result of the operation 630, the frequency domaininformation may be output. In operation 640, the image preprocessorperforms an inverse FFT (IFFT) to output image information.

In operation 650, a biological information verifying applicationverifies biological information based on the frequency domaininformation, the image information, or a combination thereof. Forexample, the biological information verifying application may verify thebiological information by comparing the frequency domain information orthe image information to registered biological information 660. Theregistered biological information 660 may correspond to the frequencydomain information or the image information. The biological informationverifying application may use an image pattern matching-basedfingerprint recognizer.

FIGS. 7 and 8 illustrate an example operation of determining aneffective area in an input image according to at least one exampleembodiment. The operation of determining an effective area in an inputimage may be performed by an image preprocessor.

Referring to FIG. 7, an input image 710 includes fingerprintinformation. The input image 710 may be adaptively cropped through themethods described with reference to FIGS. 1 through 4B. The imagepreprocessor determines an effective area 730 in a cropped image 720. Asnecessary, the image preprocessor may directly determine the effectivearea 730 in the input image 710 before the cropping.

A device 740 using the image preprocessor may determine whether torequest a new fingerprint input based on the determined effective area730. For example, when the effective area 730 is smaller than athreshold area, the device 740 may determine that the input image 710does not include sufficient fingerprint information. The device 740 maythen provide feedback to a user, for example, “please check if yourfinger is right on the input area,” which indicates the request for thenew fingerprint input.

Hereinafter, the operation of determining an effective area in an inputimage will be described in detail with reference to FIG. 8. Referring toFIG. 8, the image preprocessor sets a reference line 820 in the inputimage 720. Here, the input image 720 may be the cropped input image. Thereference line 820 may include at least one line crossing a center ofthe input image 720. For example, the reference line 820 may include twolines crisscrossing at the center of the input image 720. For ease ofdescription, the reference line 820 is set to be the two lines asillustrated in FIG. 8. However, the reference line 820 may be a singleline crossing the center of the input image 720, or at least three linescrossing the center of the input image 720.

The image preprocessor may calculate a reference energy valuecorresponding to the reference line 820. The image preprocessor maycalculate the reference energy value corresponding to the reference line820 based on pixel values of reference pixels included in the referenceline 820. For example, the reference energy value may be a variance ofthe pixels values of the reference pixels.

The image preprocessor may set a line 810 in the input image 720. Theimage preprocessor may set the line 810 using a method similar to amethod of setting an edge line. The image preprocessor may calculate anenergy value of the line 810. The energy value of the line 810 may be avariance of pixel values of pixels included in the line 810.

The image preprocessor may determine an effective area in the inputimage 720 based on the reference energy value and the energy value ofthe line 810. For example, the image preprocessor may determine whetherthe line 810 is included in the effective area by comparing the energyvalue of the line 810 to the reference energy value. When the energyvalue of the line 810 is greater than the reference energy value, theimage preprocessor may determine the line 810 to be a boundary of theeffective area.

Conversely, when the energy value of the line 810 is less than thereference energy value, the image preprocessor may increase an index ofthe line 810. The line 810 may gradually move to an opposite edgedirection, as iterations for determining the effective area in the inputimage 720 progress.

Here, in one example, when the energy value of the line 810 is equal tothe reference energy value, the same process as when the energy value isless than the reference value may be performed.

In another example, when the energy value of the line 810 is equal tothe reference energy value, the same process as when the energy value isgreater than the reference energy value may be performed.

As illustrated in FIG. 8, all the pixels included in the line 810include fingerprint information. Thus, the energy value of the line 810may be calculated to be greater than the reference energy value. In sucha case, the image preprocessor may determine the line 810 to be a leftboundary of the effective area.

Although not illustrated in FIG. 8, the image preprocessor may performthe operation described in the foregoing on a line corresponding to aright edge, a line corresponding to an upper edge, and a linecorresponding to a lower edge. Thus, the image preprocessor maydetermine a right boundary, an upper boundary, and a lower boundary ofthe effective area. The image preprocessor may determine whether apredetermined or, alternatively, desired number of boundaries, forexample, four boundaries, are determined, and iterate the operationdescribed in the foregoing until the predetermined or, alternatively,desired number of boundaries are set.

The image preprocessor may determine the effective area using thedetermined boundaries. For example, the image preprocessor may determinethe effective area using the left boundary, the right boundary, theupper boundary, and the lower boundary. The image preprocessor maydetermine whether the input image 720 includes information sufficientfor fingerprint authentication or fingerprint registration based on theeffective area. For example, when the effective area is smaller than athreshold area, the image preprocessor may determine that the inputimage 720 does not include an amount of information sufficient forfingerprint authentication or fingerprint registration.

According to an embodiment, an operation of the image preprocessor maybe implemented as an algorithm illustrated in Table 2.

TABLE 2 1. Calculate a reference variance (R1) at a center of an image,for example, a shape of +, or an entire image. Select one edge from fouredges. 2. Calculate a variance (v1) of pixel values of pixels includedin the selected edge, and compare v1 to R1. 3. When v1 is less than R1,remove an image on a line involved in the calculation of v1, and returnto operation 2. 4. When v1 is greater than R1, select another remainingedge, and return to operation 2. 5. When processing is completed at allthe edges, determine a remaining size (a width and a height) of theimage to be an effective area.

Referring to Table 2, in operation 1, the R1 may be calculated as avariance of the entire image. When the size of the effective area isless than or equal to a threshold size after the operation 5, the inputimage may be determined to be an inadequate input. In such a case, theimage preprocessor may request a user to re-input a fingerprint. Atleast some example embodiments described herein may be used as a methodof adaptively registering biological information.

FIG. 9 illustrates an example of a method of registering and verifyingbiological information using an adaptive cropping method and aneffective area determining method according to at least one exampleembodiment. Referring to FIG. 9, a biological information verifyingdevice registers biological information through operations 925, 935, and945. The biological information verifying device stores the registeredbiological information in a prearranged database 960.

In operation 925, the biological information verifying device adaptivelycrops an input image. The input image may correspond to an image to beregistered. As necessary, in operation 915, the biological informationverifying device performs image scaling on the image to be registered.For example, the biological information verifying device may enlarge orreduce a scale of the image to be registered before adaptively croppingthe image to be registered.

In operation 935, the biological information verifying device calculatesa size of an effective area, and determines whether the effective areais greater than a threshold area. In operation 945 through the operation935, the biological information verifying device enhances the image. Thebiological information verifying device may store a result image in thedatabase 960. The result image may be frequency domain information orimage information.

The biological information verifying device verifies biologicalinformation through operations 920, 930, 940, and 950. The biologicalinformation verifying device verifies the biological information usingthe registered biological information stored in the database 960.

In operation 920, the biological information verifying device adaptivelycrops an input image. The input image may correspond to an image to beverified. As necessary, in operation 910, the biological informationverifying device enlarges or reduces a scale of the input image beforeadaptively cropping the input image.

In operation 930, the biological information verifying device calculatesa size of an effective area and determines whether the effective area isgreater than a threshold area. In operation 940 through the operation930, the biological information verifying device enhances the image. Inoperation 950, the biological information verifying device verifies thebiological information by comparing a result image to the registeredbiological information stored in the database 960. The result image andthe registered biological information may be frequency domaininformation or image information.

The descriptions provided with reference to FIGS. 1 through 8 may beapplicable to the operations described with reference to FIG. 9 andthus, more detailed descriptions will be omitted here.

FIGS. 10 and 11 are flowcharts illustrating an image preprocessingmethod according to at least one example embodiment. Referring to FIG.10, one example of the image preprocessing method includes operation1010 of obtaining an input image including biological information,operation 1020 of setting edge lines in the input image, operation 1030of calculating energy values corresponding to the edge lines, andoperation 1040 of cropping the input image based on the energy values.

Referring to FIG. 11, another example of the image preprocessing methodincludes operation 1110 of obtaining an input image including biologicalinformation, operation 1120 of setting a reference line in the inputimage, operation 1130 of calculating a reference energy valuecorresponding to the reference line, operation 1140 of setting a line inthe input image, operation 1150 of calculating an energy valuecorresponding to the line, and operation 1160 of determining aneffective area in the input image based on the reference energy valueand the energy value.

The descriptions provided with reference to FIGS. 1 through 8 may beapplicable to the operations described with reference to FIGS. 10 and 11and thus, more detailed descriptions will be omitted here.

FIG. 12 illustrates an example of an electronic system according to atleast one example embodiment. Referring to FIG. 12, the electronicsystem includes a sensor 1220, a processor 1210, and a memory 1230. Thesensor 1220, the processor 1210, and the memory 1230 may communicatewith one another through a bus 1240.

The sensor 1220 may be the sensor 110 illustrated in FIG. 1. The sensor1220 may capture a fingerprint image using a well-known method, forexample, a method of converting an optical image to an electricalsignal. The captured image may be output to the processor 1210.

The processor 1210 may include or embody any or all modules describedwith reference to FIGS. 1 through 8 (e.g., the preprocessor), or performany or all operations of any or all methods described above withreference to FIGS. 1 through 8. The memory 1230 may store registeredbiological information, the input image captured by the sensor 1220, aresult image obtained by cropping the input image by the processor 1210,an effective area calculated by the processor 1210, and the like. Thememory 1230 may be a volatile memory or a nonvolatile memory.

The term ‘processor’, as used herein, may refer to, for example, ahardware-implemented data processing device having circuitry that isphysically structured to execute desired operations including, forexample, operations represented as code and/or instructions included ina program. Examples of the above-referenced hardware-implemented dataprocessing device include, but are not limited to, a microprocessor, acentral processing unit (CPU), a processor core, a multi-core processor;a multiprocessor, an application-specific integrated circuit (ASIC), anda field programmable gate array (FPGA). Processors executing programcode are programmed processors, and thus, are special-purpose computers.

The processor 1210 may execute a program, and control the electronicsystem. A program code to be executed by the processor 1210 may bestored in the memory 1230. The electronic system may be connected to anexternal device, for example, a PC and a network, through an input andoutput device (not shown), and exchange data with the external device.

The electronic system may include various forms, for example, a mobiledevice such as a mobile phone, a smartphone, a personal digitalassistant (PDA), a tablet PC, and a laptop computer, a computing devicesuch as a PC, a tablet PC, and a netbook, and a TV, a smart TV, and asecurity device for gate control.

Although at least some example embodiments herein relate mainly to acase of recognizing a user using a fingerprint of the user, at leastsome example embodiments may be expanded to a case of recognizing theuser using biological information of the user. Here, the biologicalinformation may include information about the fingerprint, informationabout blood vessels, and information about an iris of the user. In sucha case, the processor 1210 may receive, from the sensor 1220, an inputimage corresponding to the biological information of the user,adaptively crop the input image, and calculate an effective area in thecropped image.

In one example, the sensor 1220 may include a sensor configured torecognize a blood vessel pattern of the user. The sensor 1220 mayextract the blood vessel pattern from skin on a back of a hand of theuser. The sensor 1220 may maximize or, alternatively, increase abrightness of blood vessels compared to the skin using an infraredlighting and a filter, and obtain an image including the blood vesselpattern. The processor 1210 may recognize the user by comparing theinput image corresponding to the blood vessel pattern to pre-registeredblood pattern images.

In another example, the sensor 1220 may include a sensor configured torecognize an iris pattern of the user. The sensor 1220 may scan orcapture the iris pattern between a pupil and a sclera, which is whitearea of an eye, of the user. The processor 1210 may recognize the userby comparing an input image corresponding to the iris pattern topre-registered iris pattern images.

The units and/or modules described herein may be implemented usinghardware components and software components. For example, the hardwarecomponents may include microphones, amplifiers, band-pass filters, audioto digital convertors, and processing devices. A processing device maybe implemented using one or more hardware device configured to carry outand/or execute program code by performing arithmetical, logical, andinput/output operations. The processing device(s) may include aprocessor, a controller and an arithmetic logic unit, a digital signalprocessor, a microcomputer, a field programmable array, a programmablelogic unit, a microprocessor or any other device capable of respondingto and executing instructions in a defined manner. The processing devicemay run an operating system (OS) and one or more software applicationsthat run on the OS. The processing device also may access, store,manipulate, process, and create data in response to execution of thesoftware. For purpose of simplicity, the description of a processingdevice is used as singular; however, one skilled in the art willappreciated that a processing device may include multiple processingelements and multiple types of processing elements. For example, aprocessing device may include multiple processors or a processor and acontroller. In addition, different processing configurations arepossible, such a parallel processors.

The software may include a computer program, a piece of code, aninstruction, or some combination thereof, to independently orcollectively instruct and/or configure the processing device to operateas desired, thereby transforming the processing device into a specialpurpose processor. Software and data may be embodied permanently ortemporarily in any type of machine, component, physical or virtualequipment, computer storage medium or device, or in a propagated signalwave capable of providing instructions or data to or being interpretedby the processing device. The software also may be distributed overnetwork coupled computer systems so that the software is stored andexecuted in a distributed fashion. The software and data may be storedby one or more non-transitory computer readable recording mediums.

The methods according to the above-described example embodiments may berecorded in non-transitory computer-readable media including programinstructions to implement various operations of the above-describedexample embodiments. The media may also include, alone or in combinationwith the program instructions, data files, data structures, and thelike. The program instructions recorded on the media may be thosespecially designed and constructed for the purposes of exampleembodiments. Examples of non-transitory computer-readable media includemagnetic media such as hard disks, floppy disks, and magnetic tape;optical media such as CD-ROM discs, DVDs, and/or Blue-ray discs;magneto-optical media such as optical discs; and hardware devices thatare specially configured to store and perform program instructions, suchas read-only memory (ROM), random access memory (RAM), flash memory(e.g., USB flash drives, memory cards, memory sticks, etc.), and thelike. Examples of program instructions include both machine code, suchas produced by a compiler, and files containing higher level code thatmay be executed by the computer using an interpreter. Theabove-described devices may be configured to act as one or more softwaremodules in order to perform the operations of the above-describedexample embodiments, or vice versa.

Example embodiments of the inventive concepts having thus beendescribed, it will be obvious that the same may be varied in many ways.Such variations are not to be regarded as a departure from the intendedspirit and scope of example embodiments of the inventive concepts, andall such modifications as would be obvious to one skilled in the art areintended to be included within the scope of the following claims.

What is claimed is:
 1. An image preprocessing method, comprising: obtaining an input image including biological information; setting a reference line in the input image; calculating a reference energy value corresponding to the set reference line; setting a first line in the input image; calculating a first energy value corresponding to the set first line; determining an effective area in the input image based on the reference energy value and the first energy value; determining whether a first number of boundaries is determined for the effective area; and setting a second line for a boundary yet to be determined in the input image, calculating a second energy value corresponding to the second line, and determining the effective area in the input image based on the reference energy value and the second energy value, in response to a determination that the number of boundaries is not determined for the effective area.
 2. The method of claim 1, wherein the reference line comprises: at least one line passing through a center of the input image.
 3. The method of claim 1, wherein the setting of the first line comprises: setting the first line in an order starting from any one edge of the input image to an opposite edge direction, progressing iteratively to determine the effective area.
 4. The method of claim 1, wherein the reference energy value includes a variance of pixel values includes in the reference line, and the first energy value includes a variance of pixel values includes in the first line.
 5. The method of claim 1, wherein the determining comprises: determining whether the first line is included in the effective area by performing a comparison operation based on the first energy value and the reference energy value.
 6. The method of claim 1, wherein the determining comprises at least one of: determining the first line to be a boundary of the effective area in response to the first energy value being greater than the reference energy value; or setting a subsequent line in the input image, calculating a subsequent energy value corresponding to the subsequent line, and determining the effective area in the input image based on the reference energy value and the subsequent energy value, in response to the first energy value being less than the reference energy value.
 7. The method of claim 1, further comprising: determining whether to receive a re-input image based on the effective area.
 8. The method of claim 1, wherein the biological information includes at least one of fingerprint information, blood vessel information, and iris information.
 9. The method of claim 1, wherein the determining comprises: setting a subsequent line in the input image, calculating a subsequent energy value corresponding to the subsequent line, and determining the effective area in the input image based on the reference energy value and the subsequent energy value, in response to the first energy value being less than the reference energy value. 